Conversation
|
Hello! There are numerous errors with the given PR:
|
|
If there is no smime npm package available, I would strongly suggest to create one, a general package, that can be used not only in WD. |
NickOvt
left a comment
There was a problem hiding this comment.
- Tests don't pass
- Linting doesn't pass
- I would highly suggest not adding raw crypto primitives into WD codebase if possible. Is there really no smime related npm packages already? (so that the need for
smime.jsis removed). - Any refactors of WD functions (message-handler addAsync in particular, but applies to other funcs as well) is preferred to be carried out via separate branches and thus PRs or at least a single separate PR only for refactors.
- I would prefer not to have a
encryptAndPrepareMessageAsyncfunction, but rather still have two different functions. One that encrypts and the other that prepares. New functions are preferred to be lean and do one thing without side-effects.
97c37ba to
b0cda5c
Compare
They previously didn't because the linter aborted the CI, which I've now resolved. Looking at the latest CI run it seems like user creation failed, but I don't think the current changes affect that? If you could check why those tests fail, that would be great.
In general there are no good CMS libraries for Node. They're either outdated, unmaintained, fundamentally incorrect or all three at the same time. I even left a note about one of the (rather ridiculous) things I stumbled upon, if the upstream bug is fixed then it can maybe be used. In terms of making a separate package, I don't think anyone is interested in providing this as a library (especially if other people might start relying on it).
Preparing a message for encryption is a very connected process due to header manipulation. This is especially so if RFC 9788 (Header Protection) is to be implemented. While they could be split, it wouldn't provide any meaningful separation and could be a source of bugs if changed or tested separately.
Because of the previous point, only very tiny changes to addAsync and similar would stand on their own. Although if you see sections that could be pulled in separate, we can do that. Overall it was written with only PGP in mind and to add S/MIME support it had to be cleaned up a bit. |
afe6957 to
9dcd460
Compare
|
It is strongly encouraged not to force push to git branches, to have a clearer timeline of changes and diffs. |
|
That applies to general-use branches, PR/MR branches should not contain fixup or merge commits. |
|
Wildduck PRs are Squashed and Merged, there is no reason to follow the rule of force pushing to PR. Keep a clean commit history here, when PR is ready to be merged, the commits will be squashed anyway. Now it makes it difficult to follow, what exactly was changed between different commits. Last force push shows for example that nothing was changed... |
|
Squash merge has a bad habit of breaking commit signatures. |
|
There were a few tests that were dependent on each other using a PGP key that was too small, now fixed. Caused by applying same requirements on PGP as I did with S/MIME. |
|
I've fixed the last test, the old implementation exposed (and relied on) all the message headers. It is much more selective now based on the guidance of relevant RFCs with the exception of Subject should be replaced with In terms of commit log, it should be neat enough to merge without squash (that'd break the signature). (Unless something new is found and should be fixed.) |
|
I've removed the unnecessary dependency on PKI.js, it was too dangerous of a library as it caught and hid fatal exceptions. Plus a few more tests for the few things that couldn't be brought in from any other package. I've also added a guard to warn against when OpenSSL has PKCS#1 v1.5 padding disabled, because it's vital for Apple Mail. This avoids starting WildDuck in a way that causes issues. Everything works with Thunderbird and Apple Mail, testing with Outlook (New) would still be nice. |
Per WD repo policy, all PRs are squashed no matter the circumstances. |
|
Overall the PR is starting to look good. |
One is AEAD and the other isn't. That makes them very different in practice, nor are these the only (non-)AEAD algorithms that could be used. Mixing two different modes of encryption (or storage) is bad practice in cryptographic code. |
Indeed they are very different, but from code point of view, they are almost the same in output. What I mean is that, it would be great to have some sort of wrapper on top of these function or a more generic function that would be more easily extendable later on if some other algorithms are to be user or added, or KDFs changed, or other sha used, sha512 or whatever. Currently that makes it all sort of hardcoded |
|
And really, at this point the SMIMEEncryptor could just as well be moved into a separate package :) |
That's basically what Any mistakes in that logic could result in mail that can't be decrypted, so it's vital to make it as easy to follow as possible. In terms of future perspective, if anything is to be added, it's going to be a whole new set of almost everything. Like ML-KEM+SHAKE + AES-256 for post-quantum support. That would need a reasonable implementation in STD.
True, but that will likely tempt others to use it considering the sorry state of the Node ecosystem and I don't think this is a burden we want. Maybe in the future, outside the scope of this PR/MR. |
|
For reference: Content Encryption
Signature Algorithms
The reference is also why I was asking to perhaps move the SMIME encryptor into a separate package, but for now indeed let's keep it in WD codebase, it can be moved out at any time. |
|
The reference applies to MUAs and their algorithm support nor do we do any signing. Supporting smaller AES key sizes would have no practical benefit. Any hash functions besides SHA256 for MGF or KDF is simply not supported by anything. It'd really just be an easy way to generate mail that "can't" be decrypted. The current options are the only remotely viable ones up until PQ support improves in MUAs. |
After a more thorough review:Critical Issues1. Breaking change to
|
It needs an accompanying update and cleanup anyways for proper support. If it relies on a specific API, it should have (had) tests in the first place. The
That's just how headers are encrypted (based on relevant standards). All of them being exposed previously was a big flaw.
Zeroing memory that stored sensitive data is intentional. If anything subsequent fails, it should be retried with the encrypted content anyways. If any failure handling actually would need re-encrypting it should be changed.
When encryption is enabled, encryption is done, of course.
If any users exist, they should do the migration themselves and enroll new keys that aren't obsolete. Enforcing different requirements on PGP and S/MIME is not a good idea. This is just something that would have to mentioned in the changelog
Smaller keys might be forbidden by OSSL, thus the check can fail for unwanted reasons. The check is cheap enough in practice that it shouldn't be changed. One option is re-using snakeoil.key on Linux systems, but that might be more fragile in certain cases.
Suddenly faulty certificates that shouldn't have made it to user certificate storage should not cause complete encryption failures. Failing for one (new) certificate is better than for all of them.
Should be fixed now, I think this wasn't done previously either.
Correct would be to hide basically everything, but for UX reasons a minimum viable set is exposed. More is not necessary for (unencrypted) email listing - it would significantly weaken privacy protections.
S/MIME can't be enabled before disabling PGP.
Checking expiration would be a massive footgun, especially if people have a backup key in offline storage.
Generating a symmetric key for any recipient is extremely cheap, it might as well be a 100 certificates and there would be no issues.
Should be fixed. |
|
Sorry, this wasn't a discussion, but requirements to get this PR accepted.
1: No. Keep the signature of the func the same, so that existing zonemta-wildduck version does not need to be updated to work with new code. 3: Recheck all code paths just in case. Overall it seems okay-ish. Consider extreme edge cases or cases that seem impossible (things happen). 5: Explain that to users :) 6: No, check once. 13: We base of off what provides highest interoperability. 14: Either I'm blind or you really can though. You only check input params, but never whether PGP already exists on user. 15: Create a config for that. That either we allow or disallow expired certs. For any meaningful Service Provider who wants to comply to best security standards it would make sense to (at least silently) fail encryption if using expired cert. 16: The point was about added size to the message considering the volume. I will allow 16 for now as SMIME adoption (and email encryption in general) is extremely little due to it's rather bad user experience (Providers such as Protonmail and Tutanota that try to improve the situation basically create a walled garden, other than that, it's tedious to setup encryption for average email user). |
I guess, though zonemta-wildduck still needs an update. I assume you will create a PR yourself to add tests for these helper functions, if it's critical.
I don't see any that would ever re-use
If anyone at all provides this as a service then the only migration path is a manual notification anyways. In theory the encryption path can (for now) tolerate <2048-bit keys, but it'd be a hack. It's much more reasonable to note this as a "breaking" change and move on, without creating more legacy to maintain.
It basically was, but I've moved it slighly higher up for clarity.
That is the widest without unnecessarily compromising privacy and it works in practice. MUA that supports S/MIME or GPG is basically intended to work without external headers.
If it already exists they can't enable S/MIME. Even if they somehow do (bypass the API), a precedence change at that point would be expected.
Absolutely not. If the trust of the certificates is not checked, there's absolutely no point in checking expiration. Except if course when you want to create a footgun at some random date encryption gets disabled.
The certificates used aren't placed in the CMS, there's a key for each recipient identified by a SKI. Not being able to deduplicate attachments will dwarf it by a large margin. |
|
Also tested with KMail, Claws, Evolution and Trojitá. Works well. |
|
Findings
Point 3 can be skipped, I will bump the required node version in separate branch. Open Questions / Improvements
|
If the message was already plaintext and encryption fails there is no point in failing closed and to cause a service degradation. At this point in time it's not a risk that should be taken. In addition to that, anyone that desires "high-assurance" encryption should have their sender encrypt letters in the first place.
Should be better now.
Yeah, corrected. |
"Small" PR to add S/MIME at-rest encryption. With both RSA and EcDSA support. Tests for most code paths (even a few for existing PGP impl), some also cross-reference the result with
openssl. Available API for management is minimal, but should suffice. Encrypted mail can successfully be decrypted by Thunderbird, Apple Mail and Outlook (if configured to do so).In terms of what needs improvement:
encryptMessage(encryptionKey, raw, callback)that was already previously deprecated but exists forzonemta-wildduckis PGP-only and should be cleaned up.